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I A  Readiness 
Assessment  Tasking 
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ASD(C3I)  DIAP  Implementation  Plan  (12  Feb  99) 

3^  Functional  Evaluation  and  Integration  Team 
s^Consists  of  Eight  Functional  Areas 
s^Develop  lA  Performance  Goals,  Standards,  Metrics 
j=»Provide  Oversight  of  Respective  Functions 
j=»Ensure  Coherent  Integration  Throughout  DoD 
Readiness  Assessment  Function 
j=»Member  of  Functional  Evaluation  and  Integration  Team 
s^Provide  Data  Needed  to  Accurately  Assess  lA  Readiness 
J=»IA  Requirements  Identification  and  Generation 
j=»Vulnerability/Threat  Analysis,  Assessment 
3^Defense-Wide  Standards  and  Readiness  Reporting  Systems 
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DIAP 

Goal  &  Objectives 
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3^  Goal:  Ensure  a  Comprehensive,  Coherent  lA 
Program  Across  the  DoD 

3^  Objective:  Assess  the  state  of  DoD’s  lA  Posture 
3^  Tasks: 

>  Ensure  lA  assessment  is  incorporated  into  the  DoD  Exercise 
program 

Develop  business  case  and  methodology  for  lA  damage 
assessment 

Establish  appropriate  metrics 
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DIAP 

Goal  &  Objectives 
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3^  Sub-Tasks: 

5^  Develop  appropriate  and  useful  metrics 
5^  Validate  metrics 

5^  Cost  out  the  metrics  collection  processes 

Obtain  approval  from  appropriate  sources  for  metrics 
collection 

>  Deploy  DOD-wide  process  for  reporting  lA  metrics 
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Readiness  Assessment 
Goal  &  Objectives 
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^Goal:  Operationalize  lA  Readiness 
Objectives: 

®  Define  lA  Readiness  in  Operational  Context 

®  Establish  Metrics  for  Measuring  lA  Readiness 

®  Establish  Standard  Criteria  for  Applying  lA  Readiness 
Metrics 

@  Establish  lA  Readiness  Assessment  Process 

(D  Integrate  lA  Readiness  Assessment  into  Existing  DOD 
Processes 
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Challenges 
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lA  Effectiveness  is  Not  Currently  Measured 

Must  structure  lA  Readiness  Assessment  to  ensure  sufficient 
protection  of  the  information  component  of  our  war  fighting 
resources 

Must  Build  lA  assessment  into  existing  DOD  processes 

5^  lA  has  Limited  Visibility  in  the  PPBS  Process 

Must  make  lA  Readiness  fiscally  defensible 
Must  make  an  effective  Business  Case  for  lA 
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Challenges 


4 


Breaking  New  Ground  with  lA  Readiness 
Primary  Stakeholders  Must  Work  Together 

3^  Operational  Readiness  Community 
information  Assurance  Community 

Everyone’s  Looking  for  Solutions 

No  Commoniy  Accepted  lA  Metrics 
No  Commoniy  Accepted  lA  Assessment  Process 
J^Continuing  Debate  Over  Process  Review  Vs  Audit 

There  is  No  Perfect  Solution 

>  Process  Must  Inciude  Iterative  Review  and  Update 
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Challenges 


4 


2^  Information  Assurance  Readiness  Assessment  Will 
Affect  Everyone 

3^  Combat  Forces  and  Combat  Support  Agencies 

2^  Results  Must  be  Accepted  Throughout  DOD 

>  Readiness  Stakehoiders  and  iA  Stakehoiders 

>  Combat  Forces  and  Combat  Support  Agencies 
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Assessment  Framework 


Proposed  Definition  of  lA  Readiness 

“The  measured  ability  of  DOD  information  technology 
systems,  embedded  information  technologies,  and 
their  related  infrastructures  to  withstand  incidents  and 
attacks,  and  provide  effective  support  to  execution  of 
the  Department’s  combat  and  non-combat  missions.” 
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Assessment  Framework 

Concept 


Commanders-in-Chief 


Requirements 


O 


Taskings 

O 


PPBS 


Management 

Elements 
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Feedback 


/  IT  \ 

A  /Operations  M/fegg 
“i/  I  Infrastructure  / 


Feedback 


DIAP  PDIT 


I A  Readiness 
Assessment  Process 


Qualitative 
Internal  Review 
JMRR 
**  Assess  ** 


External  Review 

DIVA 

**  Validate  ** 


Quantitative 
Internal  Review 

SQRTS  Format 
**  Assess  ** 


Data 


Data 


Data 


Data 


Data  Reduction  & 
Analysis 
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Assessment  Framework 

Concept 


>-To  Ensure  Success,  Assessment  Process  Must  Be: 

3^  Consistent 

^Standard  Metrics  Should  be  Composites  to  Adequately  Measure 
“Areas  of  Interest”  Across  DoD 

^Metrics  Should  Be  Unchanging  for  Incorporation  into  Permanent 
Processes 

^  Flexible 

^Criteria  will  Apply  Standard  Metrics  Across  Diverse  Environments 
Provides  Method  to  Change  “Content”  of  Metrics,  but  not  Meaning 
^Changes  to  Criteria  Affect  Data  Considerations,  Not  Processes 
^  Relevant 

^Should  Facilitate  Analysis  to  Forecast  Capabilities,  Effectiveness 
and  Requirements 

^Metrics  are  Not  Merely  Statistics 
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Assessment  Framework 


>"Examples  of  Widely-Used  Consistent,  Flexible  Metrics: 

Dow  Jones  Industrial  Average 
5^  Gross  Domestic  Product 

>  Consumer  Price  Index 

>  Unemployment  Index 

^^Characteristics  of  Example  Metrics: 

Each  Metric  has  a  Formula,  or  Criteria,  for  Its  Application 
J^Consists  of  a  Quantity  of  Elements,  Each  with  a  Weighting  Factor 

>  Can  Nominally  Change  Each  Metric’s  Formula  Without  Changing 
Its  Meaning 

Everyone  Understands  What  the  Metrics  Represent 
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Assessment  Framework 

Concept 


>"Three  Levels  of  Criteria  Used  in  Notional  Framework: 

Department  Criteria 
^Statement  of  DoD  Policy  &  Guidance 
^Specifies  Highest-Level  Parameters  for  Metrics 
^  Service  Criteria 

^Regulations,  Instructions  Implementing  DoD  Policy 
^Applies  Service  Considerations  To  Metrics 
^  Functional  Criteria 

^Affects  Assets  Assigned  to  Support  Respective  Functional  Areas 

^Specifies  Mission-Oriented  Requirements  &  Constraints  for 
Assessed  Systems 


05  Jun  00 


15 


Aggregation  and  Analysis  of  Data 


Assessment  Framework 

Concept 


4 


t 
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Application  of  Assessment  Process 


Assessment  Framework 
Metrics  Scoring 


J^Metric  Scores  are  Same  as  in  SORTS  (C1 ,  C2,  C3,  C4) 


Ratina 

C-Ratina 

Graphic 

Excellent 

Acceptable 

C1 

C2 

O 

Marginal 

C3 

Unacceptable 

C4 

o 
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Assessment  Framework 
Metrics  Map 


Category 

Metric 

Availability 

Integrity 

Confidentiality 

Authentication 

Non-Repudiation 

People 

Adequacy  of  Critical  IT/IA  Staff  Manning  Levels 

X 

X 

X 

Adequacy  of  Critical  IT/IA  Staff  Proficiency 

X 

X 

X 

X 

X 

Adequacy  of  Security  Clearances  for  Privileged  Users 

X 

X 

X 

Effectiveness  of  Information  Systems  Security  Program 

X 

X 

X 

X 

Operations  and 
Training 

Adequacy  of  Pail  Over  Testing  for  Mission  Critical  Systems 

X 

X 

X 

Adequacy  of  Performance  Measurement  for  Network 
Infrastructure  and  Mission  Critical  Systems 

X 

X 

Effectiveness  of  Network  Penetration  Detection  and  Defense 
Capabilities 

X 

X 

X 

X 

Effectiveness  of  Network  Management  Auditing  Program 

X 

X 

X 

X 

X 

Effectiveness  of  Pirewall  Administration  Practices, 
Procedures,  and  Compliance 

X 

X 

X 

X 

X 

Adequacy  of  Requirements  for  IT  Contractor  Support 

X 

X 

X 

X 

Effectiveness  of  lA  Vulnerability  Alert  Procedures 

X 

X 

X 

X 

Equipment  and 

Adequacy  of  Technology  to  Support  Assigned  Missions 

X 

X 

X 

X 

Infrastructure 

Adequacy  of  Bandwidth  to  Support  Mission  Critical  Systems 

X 

X 

Adequacy  of  Connectivity  Robustness  for  Mission  Critical 
Systems 

X 

X 

Adequacy  and  Effectiveness  of  Survivable  Power 

X 

X 

X 

Adequacy  and  Effectiveness  of  Pacility  Security  Systems, 
Practices,  and  Procedures 

X 

X 

X 

X 

MT  i...^  r\r\ _ 

Adequacy  and  Effectiveness  of  Entry  Control  Systems  for 
Mission  Critical  and  Infrastructure  Pacilities 

X 

X 

X 

Assessment  Framework 
Notional  Metrics  Criteria 


Category 

Metric  (Aggregated) 

Metric  (Non- Aggregated) 

OSD  Criteria 

Service  Criteria 

Rating 

Criteria  for  C2  Function 

People 

Adequacy  of  lA 
Personnel  Manning 
Levels 

Adequacy  of  lA 
Personnel  Manning 
Levels 

1.  All  lA  billets 
must  be 
designated  per 
DoD  policy  xxxx 

2.  All  lA  billets 
must  be 
accounted  for 

The  following 
billets  are 
identified  as  lA 
billets 

Cl 

90%  manned,  replacements  identified  for 
outbound  personnel 

C2 

90%  manned,  replacements  not  identified  for 
outbound  personnel 

C3 

75%  to  89%  manned 

C4 

Less  than  75%  manned 

Adequacy  of  lA 
Personnel  Proficiency 

Adequacy  of  lA 
Operations  Personnel 
Proficiency 

(Maps  to  Adequacy  of 
lA  Personnel 

Proficiency  Metric) 

Operations 
personnel  must 
be  trained  and 
certified  by 
cognizant 
authority  for 
system(s)  they 
are  responsible 
for 

The  following 
billets  are 
identified  as  lA 
operations  billets 

Cl 

All  operations  personnel  have  received  xx 
hours  of  training  in  last  3  months 

C2 

All  operations  personnel  have  received  xx 
hours  of  training  in  last  6  months 

C3 

Some  personnel  have  received  no  training  in 
last  6  months 

C4 

Some  operations  personnel  are  not  certified  to 
perform  their  duties 
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Assessment  Framework 
Notional  Metrics  Criteria 


Category 

Metric  (Aggregated) 

Metric  (Non- Aggregated) 

OSD  Criteria 

Service  Criteria 

Rating 

Criteria  for  C2  Function 

People 

Adequacy  of  lA 
Personnel  Proficiency 

Adequacy  of  lA 
Maintenance  Personnel 
Proficiency 

(Maps  to  Adequacy  of 
lA  Personnel 

Proficiency  Metric) 

Maintenance 
personnel  must 
be  trained  and 
certified  by 
cognizant 
authority  for 
system(s)  they 
are  responsible 
for 

The  following 
billets  are 
identified  as  lA 
maintenance 
billets 

Cl 

80%  or  more  of  assigned  lA  maintenance 
personnel  are  mid-skill  level  qualified  or  above 

C2 

70%  or  more  of  assigned  lA  maintenance 
personnel  are  mid-skill  level  qualified  or  above 

C3 

60%  or  more  of  assigned  lA  maintenance 
personnel  are  mid- skill  level  qualified  or  above 

C4 

Less  than  60%  of  assigned  lA  maintenance 
personnel  are  mid- skill  level  qualified  or  above 

Adequacy  of 

Information  Systems 
Security  Office 

Personnel  Proficiency 

(Maps  to  Adequacy  of 
lA  Personnel 

Proficiency  Metric) 

IS  SO  personnel 
must  be  trained 
and  certified  by 
cognizant 
authority 

The  following 
billets  are 
identified  as  ISSO 
billets 

Cl 

All  assigned  ISSO  personnel  have  completed 
formal  training  and  been  certified 

C2 

Some  assigned  ISSO  personnel  have 
completed  formal  and/or  informal  training  but 
not  been  certified 

C3 

Some  assigned  ISSO  personnel  have 
completed  no  formal  and/or  informal  training 

C4 

No  assigned  ISSO  personnel  completed  any 
formal  and/or  informal  training 
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Assessment  Framework 
Notional  Metrics  Criteria 


Category 

Metric  (Aggregated) 

Metric  (Non- Aggregated) 

OSD  Criteria 

Service  Criteria 

Rating 

Criteria  for  C2  Eunction 

People 

Adequacy  of  Security 
Clearances  for 
Privileged  Users 

Adequacy  of  Security 
Clearances  for 

Privileged  Users 

1 .  All  privileged 
user  billets  must 
be  designated  per 
DoD  policy  xxxx 

2.  All  privileged 
users  must  be 
cleared  for  the 
classification  of 
the  system  they 
have  access  to 

The  following 
billets  are 
identified  as 
privileged  user 
billets/positions 

Cl 

TBD 

C2 

TBD 

C3 

TBD 

C4 

TBD 

Effectiveness  of 
Information  Systems 
Security  Program 

Effectiveness  of 
Information  Systems 
Security  Program 

Each  IS  Security 
Program  must 
have  a  charter 
explicitly 
promulgated  by 
the  installation 
commander  or 
equivalent 

TBD 

Cl 

TBD 

C2 

TBD 

C3 

TBD 

C4 

TBD 
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Assessment  Framework 
Notional  Metrics  Criteria 


Category 

Metric  (Aggregated) 

Metric  (Non- Aggregated) 

OSD  Criteria 

Service  Criteria 

Rating 

Criteria  for  C2  Function 

Operations 
and  Training 

Adequacy  of 
Automatic/Manual 

Fail  Over  Testing  for 
Mission  Critical 
Systems 

Adequacy  of 
Automatic/Manual  Fail 
Over  Testing  for 

Mission  Critical 

Systems 

All  mission 
critical  back-up 
systems  must 
provide 
capabilities  as 
designed,  as 
required,  and 
within  applicable 
constraints 

The  following 
systems  are 
designated  as 
mission  critical 
back-up  systems 

Cl 

All  systems  have  auto-fail  over  capability  and 
were  tested  successfully  within  the  last  month 

C2 

All  systems  have  auto-fail  over  capability  and 
were  tested  successfully  within  the  last  2 
months 

C3 

All  systems  have  auto-fail  over  capability  and 
were  tested  successfully  within  the  last  3 
months 

C4 

All  systems  have  auto-fail  over  capability  and 
were  not  tested  successfully  within  the  last  3 
months,  or  some  have  no  auto-fail  over 
capability 

Adequacy  of 
Performance 
Measurement  for 
Network 

Infrastructure  and 
Mission  Critical 
Systems 

Adequacy  of 
Performance 
Measurement  for 
Network  Infrastructure 
and  Mission  Critical 
Systems 

All  systems  must 
meet  or  exceed 
operational 
availability 
requirements 

Reference 

applicable  system 

requirements 

documents  for 

operational 

availability 

requirements 

Cl 

TBD 

C2 

TBD 

C3 

TBD 

C4 

TBD 
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Assessment  Framework 
Notional  Metrics  Criteria 


Category 

Metric  (Aggregated) 

Metric  (Non- Aggregated) 

OSD  Criteria 

Service  Criteria 

Rating 

Criteria  for  C2  Eunction 

Operations 
and  Training 

Effectiveness  of 
Network  Penetration 
Detection  and 

Defense  Capabilities 

Effectiveness  of 

Network  Penetration 
Detection  and  Defense 
Capabilities 

Procedures  must 
be  in  place  &  use 
to  respond  to  and 
report  network 
penetration 
activities 

Cl 

TBD 

C2 

TBD 

C3 

TBD 

C4 

TBD 

Effectiveness  of 
Network 

Management 

Auditing  Program 

Effectiveness  of 

Network  Management 
Auditing  Program 

Procedures  must 
be  in  place  &  use 
for  Continuity  of 
Ops;  disaster 
recovery 
planning;  risk 
detection  & 
mitigation;  use  of 
updated  software 
patches;  and  use 
of  updated  anti¬ 
virus  software 
and  signatures 

Cl 

Perform  random  audits  to  measure  network 
security  policy  compliance.  100%  of  nets  have 
been  audited  in  last  year,  25%  in  last  3  months 

C2 

Perform  scheduled  audits  to  measure  network 
security  policy  compliance.  100%  of  nets  have 
been  audited  in  last  year,  25%  in  last  3  months 

C3 

Perform  random  or  scheduled  audits  to 
measure  network  security  policy  compliance. 

LT  100%  of  nets  have  been  audited  in  last 
vear,  with  25%  occurring  in  last  3  months 

C4 

Less  than  25%  of  installation  nets  have  been 
audited  in  last  3  months 
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Assessment  Framework 
Notional  Metrics  Criteria 


Category 

Metric  (Aggregated) 

Metric  (Non- Aggregated) 

OSD  Criteria 

Service  Criteria 

Rating 

Criteria  for  C2  Function 

Operations 
and  Training 

Effectiveness  of 
Firewall 
Administration 
Practices,  Procedures, 
and  Compliance 

Effectiveness  of 

Firewall  Administration 
Practices,  Procedures, 
and  Compliance 

Firewalls  must 
not  be  in  factory 
default 
configuration 

Cl 

Duties  performed  by  dedicated  personnel  with 
formal  training 

C2 

Duties  performed  as  extra-duty  by  personnel 
with  formal  training 

C3 

Duties  performed  by  dedicated  personnel 
without  formal  training 

C4 

Duties  performed  as  extra-duty  by  personnel 
without  formal  training 

Adequacy  of 
Requirements  for 
Contractor  Support 

Adequacy  of 
Requirements  for 
Contractor  Support 

Consideration 
must  be  given  to 
the  following 
contractual  items: 

Response  times, 

minimum 

qualifications, 

performance 

guarantees, 

security 

clearances,  etc.. 

Cl 

TBD 

C2 

TBD 

C3 

TBD 

C4 

TBD 
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Assessment  Framework 
Notional  Metrics  Criteria 


Category 

Metric  (Aggregated) 

Metric  (Non- Aggregated) 

OSD  Criteria 

Service  Criteria 

Rating 

Criteria  for  C2  Eunction 

Operations 
and  Training 

Effectiveness  of 
Information 

Assurance 
Vulnerability  Alert 
Procedures 

Effectiveness  of 
Information  Assurance 
Vulnerability  Alert 
Procedures 

All  DoD 
elements  must 
comply  with 
lAVA 

compliance  and 

reporting 

requirements 

Cl 

All  required  actions  have  been  accomplished, 
and  100%  were  within  time  constraints 

C2 

All  required  actions  have  been  accomplished, 
and  80%  were  within  time  constraints 

C3 

All  required  actions  have  been  accomplished, 
and  60%  were  within  time  constraints 

C4 

All  required  actions  have  not  been 
accomplished 

Equipment 

and 

Infrastructure 

Adequacy  of 
Technology  to 

Support  Assigned 
Mission 

Adequacy  of 

Technology  to  Support 
Assigned  Mission 

Consideration 
must  be  given  to 
the  following 
items: 

Age  of 

equipment;  and 
age,  capability, 
robustness  of 
crypto,  etc.. 

Cl 

TBD 

C2 

TBD 

C3 

TBD 

C4 

TBD 
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Assessment  Framework 
Notional  Metrics  Criteria 


Category 

Metric  (Aggregated) 

Metric  (Non- Aggregated) 

OSD  Criteria 

Service  Criteria 

Rating 

Criteria  for  C2  Function 

Equipment 

and 

Infrastructure 

Adequacy  of 
Bandwidth  to  Support 
Assigned  Mission 
Critical  Systems 

Adequacy  of  Bandwidth 
to  Support  Assigned 
Mission  Critical 

Systems 

All  DoD 
elements  must 

measure 

bandwidth  for  all 
assigned  systems 
that  compete  with 
mission  critical 
systems  for 
bandwidth 

resources 

The  following 
systems  are 
designated 

Mission  Critical 
Systems 

Cl 

Installation  has  sufficient  bandwidth  such  that 
normal  utilization  consumes  a  max  of  40%  and 
projected  surge  is  less  than  70%  of  available 

C2 

Installation  has  sufficient  bandwidth  such  that 
normal  utilization  consumes  a  max  of  60%  and 
projected  surge  is  less  than  80%  of  available 

C3 

Installation  has  sufficient  bandwidth  such  that 
normal  utilization  consumes  a  max  of  80%  and 
projected  surge  is  less  than  90%  of  available 

C4 

Normal  utilization  consumes  more  than  80%  or 
projected  surge  is  greater  than  90%  of 
available 

Adequacy  of 
Connectivity 
Robustness  to  Support 
Assigned  Mission 
Critical  Systems 

Adequacy  of 
Connectivity  Robustness 
to  Support  Assigned 
Mission  Critical 

Systems 

TBD 

TBD 

Cl 

All  systems  have  dual  circuits  available,  with 
dual  routing,  and  no  known  single  points  of 
failure 

C2 

All  systems  have  dual  circuits  available,  with 
dual  routing,  and  known  single  points  of  failure 

C3 

All  systems  have  dual  circuits  available, 
without  dual  routing 

C4 

One  or  more  systems  are  single  threaded 
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Assessment  Framework 
Notional  Metrics  Criteria 


Category 

Metric  (Aggregated) 

Metric  (Non- Aggregated) 

OSD  Criteria 

Service  Criteria 

Rating 

Criteria  for  C2  Eunction 

Equipment 

and 

Infrastructure 

Adequacy  and 
Effectiveness  of 
Survivable  Power 

Adequacy  and 
Effectiveness  of 
Survivable  Power 

TBD 

TBD 

Cl 

Systems  have  auto-switching  power  that  tested 
successfully  in  last  3  months 

C2 

Systems  have  auto-switching  power  that  tested 
successfully  in  last  6  months 

C3 

Systems  have  auto-switching  power  that  tested 
unsuccessfully  in  last  6  months 

C4 

Systems  have  auto- switching  power  not  tested 
in  last  6  months,  or  no  auto- switching  power 

Adequacy  and 
Effectiveness  of 
Eacility  Security 
Systems,  Practices, 
and  Procedures 

Adequacy  of 
Connectivity  Robustness 
to  Support  Assigned 
Mission  Critical 

Systems 

TBD 

TBD 

Cl 

Eacility  is  patrolled,  fenced,  lighted,  and  has 
intrusion  alarm  system 

C2 

Eacility  is  patrolled,  fenced,  and  lighted 

C3 

Eacility  is  patrolled  and  fenced 

C4 

Eacility  has  no  perimeter  protection 

Adequacy  and 
Effectiveness  of  Entry 
Control  Systems  for 
Mission  Critical  and 
Infrastructure 

Eacilities 

Adequacy  and 
Effectiveness  of  Entry 
Control  Systems  for 
Mission  Critical  and 
Infrastructure  Eacilities 

TBD 

TBD 

Cl 

TBD 

C2 

TBD 

C3 

TBD 

C4 

TBD 
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Activities 

Current  &  Forthcoming 


4 


3^  Continue  Development  of  Strawman  Framework 
3^  Readiness  Assessment  Workshop  12-14  July  2000 

5^  DoD  Agency,  Service,  Joint  Staff  and  CiNC  Participation 
>  Formaiize  Assessment  Framework 

3^  Draft  Implementing  Guidance 
3^  Beta  Test 
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lA  Readiness  Assessment 
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